Picture detecting sensor unit

ABSTRACT

A picture detecting sensor unit has a passive sensor resolving a field of view into picture elements having different brightness. Picture processing means receive data from the picture detecting sensor. An additional active LADAR sensor having a laser emitting a laser beam and a scanning system for scanning an area of the field of view by this laser beam provides distance signals from the radiation reflected by an object in the field of view. The scanning of the field of view by the LADAR sensor is controlled by the picture processing means associated with the passive sensor such that the scanning effected by the LADAR sensor is limited to objects detected by the passive sensor in the field of view. A common imaging optical system is provided for the LADAR sensor and the passive sensor. A beam splitter separates the beams detected by the passive sensor and the returning beams of the LADAR sensor.

The invention relates to a picture detecting sensor unit comprising apassive sensor resolving a field of view into picture elements havingdifferent brightness, picture processing means being arranged to receivedata therefrom.

Picture detecting sensor units are known, in which the individualpicture elements of an observed field of view are imaged successively bymeans of movable optical elements onto a stationary detector. The fieldof view is thus picked-up point by point or row by row. Also sensorshaving a two-dimensional array of detector elements are known. Suchsensors may be formed, e.g. by a CCD matrix. In such sensors, the fieldof view is imaged onto the detector elements of the sensor. The detectorelements respond to radiation emitted or reflected by the objects of thefield of view. The sensor itself does not emit any radiation,differently to e.g. RADAR. Such sensors are called "passive" picturedetecting sensors.

Such passive sensors provide a light/dark pattern only, in whichintensity values of the radiation falling on the sensor from therespective observation directions are associated with the differentobservation directions. The passive sensors do not provide any depthinformation, that is no information about the distance or thethree-dimensional structure of a detected object.

Furthermore, it is known to combine a passive picture detecting opticalsensor with an "active" sensor operating in the millimeter wavelengthrange. Such a sensor provides distance information. However the angularresolution of such a sensor operating in the millimeter wavelength rangeis small as compared to an optical picture detecting sensor. Thereforeit is not or only to an insufficient extent possible to geometricallyassociate the informations of such a sensor with the individual pictureelements. Furthermore, such combined sensor units are undesirably bulky.

Furthermore, LADAR sensors are known. Following RADAR, LADAR means,"Laser Detection and Ranging". Such sensors comprise a laser emitting alaser light beam. The laser light beam falls onto an object detected inthe field of view. A photoelectric detector detects the light reflectedby the object. The distance of the object can be determined from thetravel time of the light beam to the object and back.

It is the object of the invention to provide a sensor unit permittingimproved recognition of objects in an observed field of view.

It is a further object of the invention to provide a compact "dual-mode"sensor unit.

According to the invention, the indicated objects are achieved by apicture detecting sensor unit of the type mentioned in the beginningcomprising a passive sensor resolving a field of view into pictureelements having different brightness, and picture processing meansarranged to receive data therefrom. The invention provides an additionalactive LADAR sensor having a laser emitting a laser beam and means forproviding distance signals from the radiation of said laser beam whichis reflected by surfaces in the field of view.

The invention thus provides a combination of a picture detecting opticalsensor with a LADAR sensor. By the additional use of the LADAR sensor,also the distance of the objects can be determined in addition to thetwo-dimensional structure thereof. Information about the absolutemagnitude of a detected object can be obtained therefrom. When the fieldof view is scanned by a laser light beam, it is also possible to obtaininformation about the Three-dimensional structure of the object, that isa kind of relief. The scanning by means of a laser light beam permitssufficiently high resolution in contrast to sensors operating withmillimeter waves.

The LADAR sensor may operate either as FMCW-LADAR or as pulse-LADAR.

If the field of view is scanned by means of the LADAR-sensor with thesame resolution, e.g. in a 512×512 picture matrix, and at the same framefrequency of e.g.. 25 frames per second as this is done with the passiveoptical sensor, this will result in a very high scanning frequency ofe.g.. 6,5 megacycles per second. Such a scanning frequency is difficultto achieve with a LADAR sensor. In accordance with a furthermodification of the invention, the picture detecting sensor unit furthercomprises scanning means for scanning the field of view with the laserbeam of said LADAR sensor, said scanning means being controlled by saidpicture processing means receiving data from said passive sensor.

Preferably, said scanning means of said LADAR sensor are controlled tolimit the scanning effected by the LADAR sensor to objects detected inthe field of view by said passive sensor.

The picture processing means may comprise segmentation means for thesegmentation of objects detected by said sensor in the field of view,the LADAR sensor being controlled to scan such segmented objects only.

If the LADAR sensor is controlled this way, the sensor needs not have toscan the whole field of view at the frame frequency of e.g.. 25 cyclesper second. This field of view may contain an object to be identified ina small area only. The rest of the field of view is background withoutinterest. Thus, at first, a selection is made by the passive sensor andthe associated picture processing, which partial area of the field ofview is worth considering for scanning by the LADAR sensor. Such apartial area forms e.g. a picture matrix of 16×16. Considerably smallerscanning frequencies in the range of 10 kilocycles per second aresufficient for the scanning of such a limited partial area at the framefrequency of 25 frames per second. Such scanning frequencies are easy tohandle.

The scanning of the objects by means of the LADAR-sensor can beexpedited in that said scanning means of said LADAR-sensor arecontrolled to scan, in a first scanning cycle, the objects detected bysaid passive sensor along a path sweeping over the objects ony once,whereby first scanning cycle information is obtained, and to scan, in asecond scanning cycle, line-by-line limited areas of the field of view,each of which contains one of said objects, and said first scanningcycle information is applied to selecting means for selecting, from saidobjects detected by said passive sensor, those objects meetingpredetermined criteria, said scanning means of said LADAR-sensor beingcontrolled by said selecting means to scan, during said second scanningcycle, only the areas containing the objects thus selected.

In order to make a compact and space saving sensor unit., a commonimaging optical system is provided for the LADAR sensor and for thepassive sensor, a beam splitter being provided for separating the beamsfalling on the passive sensor and the returning beams of the LADARsensor.

Two embodiments of the invention are described in further detailhereinbelow with reference to the accompanying drawings.

FIG. 1 shows the construction of a sensor unit having a passive picturedetecting optical sensor and a LADAR sensor which both operate with acommon imaging optical system and in which the LADAR sensor iscontrolled by the picture processing of the passive sensor.

FIG. 2 is a schematic illustration of another embodiment of the sensorunit and a block diagram which illustrates schematically the variousfunctions of the sensor unit.

FIG. 3 shows the scanning of the field of view by the scanning beam ofthe LADAR-sensor in a first scanning cycle controlled by the signals ofthe passive sensor.

FIG. 4 shows the scanning of the field of view by the scanning beam ofthe LADAR-sensor in a second scanning cycle controlled by the signals ofthe passive sensor.

FIG. 5 is a schematic illustration and shows, one on top of the other,the informations which are obtained about the object by means of thepassive sensor and by means of the LADAR-sensor during the first andsecond scanning cycles.

FIG. 6 shows the sequence of the first and second scanning cycles of theLADAR-sensor.

In FIG. 1, numeral 10 designates a structure on which the sensor unit ismounted. The structure 10 has a flange 12. An optical system 14 ismounted on the flange 12. The optical system 14 is common to both thepassive picture detecting optical sensor and to the LADAR sensor anddefines a common aperture for both sensors.

A wave length selective beam splitter 16 separates the paths of rays ofthe passive picture detecting optical sensor and of the LADAR sensor.The path of rays of the passive sensor is reflected by the beam splitter16 and directed by a deflecting mirror 18 and a further optical element20 onto a detector 22. The detector 22 is a CCD detector (Charge CoupledDevice) having a two-dimensional array of detector elements forming a512×512 matrix. However, the detector may be, instead, also a linedetector having an one-dimensional array of detector elements. In thiscase the deflection mirror 18 is a vibrating mirror causing the field ofview to be scanned row by row.

The path of rays of the LADAR sensor passes through the wave lengthselective beam splitter 16. A scanning system 30 is provided in the pathof rays. The scanning system comprises a mirror, which can becontrollably rotated about two mutually orthogonal axes through definedangles. The path of rays--on the way back--passes through a quarter waveplate 32, through a deflecting mirror 34 and through a polarisation beamsplitter 36. The light beam passing through the polarisation beamsplitter 36 is focused by an optical element 38 on a laser detector 40.

A laser light beam coupled out of the laser 50 is linearily polarized.This laser light beam is directed to a deflecting mirror 52 anddeflected thereby. The laser light beam is enlarged by optical system54. A deflecting mirror 56 then deflects the laser light beam towardsthe polarisation beam splitter 36. The linearily polarized laser lightbeam is incident on the polarisation beam splitter 36 at the Brewsterangle. Thereby, the outgoing laser light beam is coupled into the pathof rays without loss.

After having passed through the quarter wave plate 32 the outgoing laserlight beam is circularly polarized. The laser light beam is deflected bythe scanning system 30, passes through the wave length selective beamsplitter 16 and emerges through the optical system 14. The returninglaser light beam reflected by an object is also circularly polarized.The returning laser light beam runs the same way back through theoptical system 14, through the scanning system 30, through the wavelength selective beam splitter 16 and the quarter wave plate 32. Afterhaving passed through the quarter wave plate 32, the laser light beam isagain linearily polarized. However, the polarisation plane of thereturning laser light beam is rotated by 90° as compared to thepolarisation plane of the outgoing laser light beam. The polarisationbeam splitter 36 has a high transmission for the thus polarized laserlight beam. Thereby, the returning laser light beam can reach the laserdetector 40 through the polarisation beam splitter 36 substantiallywithout loss.

The picture data of the detector 22 of the passive sensor are applied toa picture processing device 60. The picture processing device 60 causessegmentation of individual objects and selects individual objects ofinterest in the field of view. The picture processing device controlsthe scanning system 30 such that it scans a partial area of the field ofview containing such an object of interest. Such a partial area maycover e.g.. 16×16 picture elements or pixels. Such a partial area isconsiderably smaller than the whole field of view. Thus it can bescanned at a considerably lower scanning frequency of e.g.. 10kilocycles per second than the frequency which would be necessary forscanning the whole field of view at the same frame frequency.

FIG. 2 shows schematically the design of a modified sensor unit, thevarious functions being illustrated in the form of a block diagram.

The sensor unit of FIG. 2 has an optical system 64, which is, again,common to the passive sensor and to the LADAR-sensor. In the embodimentof FIG. 2, the optical system 64 comprises an annular concave mirror 66,which faces the observed field of view, and a convex secondary mirror68. The secondary mirror faces the concave mirror 66 and receives thelight beam 70 focussed by the concave mirror 66. The secondary mirror 68reflects this light beam 70 along the optical axis 72 of the concavemirror 66 through a central opening of the concave mirror 66. Apartially transparent mirror 74 is arranged in the converging part ofthe beam behind this central opening and at an angle of 45° with respectto the optical axis 72.

The path of rays is divided by the partially transparent mirror 74 intoa path of rays 76 of the passive sensor and a path of rays 78 of theLADAR-sensor.

In the embodiment of FIG. 2, the passive sensor has linear detectorarray, which scans a field of view two-dimensionally via an oscillatingmirror. The detector array consists of a linear array ofinfrared-sensitive detector elements. In FIG. 3, the detector array inthe image plane of the imaging optical system 64 has the referencenumeral 80. The scanning of the field of view via an oscillating mirroris symbolized by a double arrow 82 in FIG. 3. In FIG. 2, the detectorarray 80 with the oscillating mirror are generally illustrated as anIR-scanner 84.

The IR-scanner 84 of the passive sensor provides a two-dimensionalpicture of the field of view. This picture is processed by a pictureprocessing device 86. The picture processing device 86 comprisessegmenting means for the segmentation of the individual objects observedin the field of view. Such objects are illustrated in FIGS. 3 and 4 andare designated by numerals 88, 88A and 88B.

The path of rays 78 of the LADAR-scanner includes a negative lens 90.The negative lens makes the outgoing, collimated laser light beam 92 ofthe LADAR-sensor divergent, and recollimates the returning laser beam.The laser beam is generated by a laser 94 designed as a CO₂ -pulselaser. A LADAR-picture scanning system 96 is arranged to deflect, in twodimensions, the laser light beam generated by the laser 94. Thereby, anarea can be scanned, as schematically shown in FIG. 4. The LADAR-picturescanning system 96 can have a mirror arranged to be deflected about twoaxes, similar to the mirror of the LADAR-picture scanning system 30 ofFIG. 1. The LADAR-picture scanning system 96 provides signals which areapplied to a LADAR-signal processing unit 98. The LADAR-signalprocessing unit 98 provides distance information, each distanceinformation being associated with a particular location in the field ofview of the LADAR-picture scanning system 96.

The LADAR-picture scanning system 96 is controlled by the signalprocessing device 86 of the passive sensor. To this end, the deflectionsof the mirror of the LADAR-picture scanning system 96 are commanded.This is illustrated in FIG. 2 by the connecting line 100. The control ofthe LADAR-picture scanning system 96 will be described hereinbelow withreference to FIGS. 3 to 6.

The picture information from the picture processing device 84 and thedistance information from the LADAR-signal processing unit 98 areapplied to a further signal processing unit 102. This is illustrated inFIG. 2 by the connecting lines 104 and 106, respectively. The signalprocessing unit 102 determines therefrom a three-dimensionalrepresentation of a detected object 88 or 88A or 88B. From thisthree-dimensional representation, the signal processing unit establisheswhich of the objects 88, 88A and 88B can be regarded as a "target". Aguided missile may, for example, directed to this target.

A gimbal scanning device 108 superimposes the scanning of a larger fieldof view by rotating the whole sensor unit of FIG. 2 in a gimbal (notshown). This is schematically represented in FIG. 3 by an arrow 110.

The scanning of the field of view by the passive sensor, i.e. the lineardetector array 80, can be seen best from FIG. 3. The detector array 80extends in horizontal direction in FIG. 3 and scans the field of viewtransversely thereto, i.e. vertically in FIG. 2. This is schematicallyshown by the double arrow 82. A scanning of the field of viewlongitudinally of the detector array is superimposed to this scanningmovement. This longitudinal scanning is caused by the gimbal scanningdevice 108 and is schematically illustrated by the arrow 110 in FIG. 3.Correspondingly, the field of view is scanned along a zigzag path 112.The scanning movement caused by the oscillating mirror and representedby the double arrow 82 and the gimbal scanning movement represented byarrow 110 are matched such that--in the image plane--during one cycle ofthe back-and-forth scanning movement represented by double arrow 83, thegimbal scanning movement represented by arrow 110 has advanced by aboutthe length of the detector array 80.

The scanning movement of the LADAR-picture scanning system 96 is limitedto those regions of the field of view in which the passive sensor hasdetected objects 88 at all. Furthermore, scanning is effected in twoscanning cycles. A first scanning cycle seves to eliminate, from theobjects detected by the passive sensor, those objects which cannot beregarded as valid targets from the beginning. This may, for example, beobjects which do not show a three-dimensional structure.

In the first scanning cycle, the objects 88, 88A, 88B are scanned alonga straight path 114 extending once across the object 88, 88A or 88B.From the path 114 of a first object 88, the LADAR-picture scanningsystem 96 jumps, along the path 116 shown by a dashed line, to theregion of the next object 88A. The object 88A is then scanned againalong a straight path 114A extending once across the object 88A. Fromthe region of the object 88A, the LADAR-picture scanning system 96 jumpsto the region of the object 88B along the path 116A shown by a dashedline. The object 88B is scanned along a straight path 114B extendingonce across the object 88B.

Thanks to this relatively quick "linear" scanning of all detectedobjects, individual objects such as object 88A can be excluded frombeing regarded a valid target. Then the complete scanning by theLADAR-picture scanning system 96 needs only be carried out, in a secondscanning cycle, with the remaining objects 88 and 88B. This isillustrated in FIG. 4. The complete scanning of the objects 88B and 88is effected in a second scanning cycle during the return run of thegimbal scanning movement. This scanning is effected by scanning alwaysonly a limited area 118B, 118 containing the respective object. Thisarea is scanned along a zigzag path 120B and 120, respectively. Betweenthese zigzag paths 120B and 120, the LADAR-picture scanning system 96jumps along a path 122 from the area 118B to the area 118. The object88A is spared.

The LADAR-sensor operates more slowly than the passive sensor. Thanks tothe limitation of the areas scanned by the LADAR-sensor and theselection of the objects detected by the passive sensor, the informationfrom the LADAR-sensor can, nevertheless, be available simultaneouslywith the information from the passive sensor. The signal processing unit102 provides a three-dimensional representation of the object.

In FIG. 5, the various informations about the object, for example object88, are schematically illustrated.

A pixel matrix 124 is shown in the upper portion of FIG. 5. In thispixel matrix 124, the field of view of the passive sensor is subdividedinto a raster of picture elements (pixels). In this pixel matrix 124, anobject 88 appears as a two-dimensional pattern 126.

After the object 88 has been scanned in the first scanning cycle, asillustrated in FIG. 3, information is available about the distance ofthose individual portions of the object 88, which are located on thestraight, one-dimensional path 114. This provides a profile 128 asillustrated in the median part of FIG. 5. Thereby the object 88 hasqualified as possible target.

The object 88 is then scanned two-dimensionally as shown in FIG. 4. Thistwo-dimensional scanning provides a complete three-dimensionalrepresentation 130 of the surface of the object 88. Based on thisrepresentation, a certain type of object may then be selected as target.

FIG. 6 illustrates the sequence of the scanning by means of theLADAR-sensor.

At first, there is a first scanning cycle as shown in FIG. 3. This isrepresented by rectangle 132. Based on this first scanning cycle, aselection is made. This is represented in FIG. 6 by rectangle 134. Thesecond scanning cycle as shown in FIG. 4 is now applied to the selectedobjects only. This second scanning cycle is represented by rectangle136. The signal processing unit 102 now evaluates the data to determinethe three-dimensional structure of the object. This is represented byrectangle 138. Eventually, a target is identified on the basis of thestructure thus determined. This is represented by rectangle 140.

In certain cases, no scanning may be necessary at all. It might besufficient to determine just the distance of the detected object. Thisdistance would permit, together with the picture at the detector 22, astatement about the absolute magnitude of the object.

I claim:
 1. A device for detecting and identifying an object in a fieldof view comprising:laser means for generating a laser beam and means fordirecting said laser beam towards said object; scanning beam deflectingmeans for deflecting said laser beam to scan two-dimensionally over allof the surface of said object; LADAR sensor means exposed to laser lightreflected by said object for providing relief information for eachscanned spot of said object; optical system means for forming an imageof said field of view in an image plane; passive image resolving sensormeans in said image plane for providing picture output signalsindicative of the two-dimensional image of said field of view; beamsplitter means between said optical system means and said image planefor directing said laser beam deflected by said scanning beam deflectingmeans through said optical system means towards said object and fordirecting said reflected laser light towards said LADAR sensor means;image processing means to which said picture output signals from saidpassive image resolving sensor means are applied for segmenting anddetecting individual objects within said field of view, said imageprocessing means being arranged to control said scanning beam deflectingmeans such that said laser beam scans substantially only the surface ofobjects detected by said image processing means within said field ofview; means for deriving, from said relief information, athree-dimensional relief representation of said object; and means foridentifying said object on the basis of said three-dimensional reliefrepresentation.
 2. A device for detecting and identifying an object in afield of view comprising:laser means for generating a laser beam andmeans for directing said laser beam towards said object; scanning beamdeflecting means for deflecting said laser beam to scantwo-dimensionally over all of the surface of said object; LADAR sensormeans exposed to laser light reflected by said object for providingrelief information for each scanned spot of said object; optical systemmeans for forming an image of said field of view in an image plane;passive image resolving sensor means in said image plane for providingpicture output signals indicative of the two-dimensional image of saidfield of view; beam splitter means between said optical system means andsaid image plane for directing said laser beam deflected by saidscanning beam deflecting means through said optical system means towardssaid object and for directing said reflected laser light towards saidLADAR sensor means; image processing means to which said picture outputsignals from said passive image resolving sensor means are applied forsegmenting and detecting individual objects within said field of view,said image processing means being arranged to control said scanning beamdeflecting means such that said laser beam scans substantially only thesurface of objects detected by said image processing means within saidfield of view; wherein said scanning beam deflecting means arecontrolled by said image processing means to scan, in a first scanningcycle, the objects detected by said passive image resolving sensor alonga path sweeping over the objects only once, whereby first scanning cycleinformation is obtained, said first scanning cycle information isapplied to selecting means for selecting, out of the objects detected bysaid passive image resolving sensor those objects meeting predeterminedcriteria, and said scanning beam deflecting means being controlled bysaid image processing means and said selecting means to scan, in asecond scanning cycle, line-by-line, only limited areas of the field ofview containing said selected objects; means for deriving, from saidrelief information, a three-dimensional relief representation of saidobject; and means for identifying said object on the basis of saidthree-dimensional relief representation.
 3. A device as claimed in claim2, wherein said beam splitter is a wavelength-selective beam splitter.4. A device as claimed in claim 3, wherein said scanning beam deflectingmeans are arranged between said beam splitter and said LADAR sensormeans.
 5. A device as claimed in claim 4, wherein a quarter wave plateand a polarization beam splitter are provided between said beamdeflecting means and said LADAR sensor means, said quarter wave plateand polarization beam splitter being arranged to reflect said outgoinglaser light beam from said laser means towards said optical system meansand to pass said reflected laser light to a detector of said LADARsensor means.
 6. A device for detecting and identifying an object in afield of view comprising:optical system means for forming an image ofsaid field of view in an image plane; passive image resolving sensormeans in said image plane for providing picture output signalsindicative of the two-dimensional image of said field of view; lasermeans for generating a laser beam; scanning beam deflecting means fordeflecting said laser beam, LADAR sensor means; beam splitter meansbetween said optical system means and said image plane for directingsaid laser beam deflected by said scanning beam deflecting means throughsaid optical system means towards said object to scan over the wholesurface of said object and for directing light reflected by said objectand collected by said optical system means towards said LADAR sensormeans for providing relief information for each scanned spot of saidobject; image processing means to which said picture output signals fromsaid passive image resolving sensor means are applied for segmenting anddetecting individual objects within said field of view, said imageprocessing means being arranged to control said scanning beam deflectingmeans such that said laser beam scans substantially only the surface ofobjects detected by said image processing means within said field ofview; means for deriving, from said relief information, athree-dimensional relief representation of said object; and means foridentifying said object on the basis of said two-dimensional image andsaid three-dimensional relief representation.
 7. A device for detectingand identifying an object in a field of view comprising:optical systemmeans for forming an image of said field of view in an image plane;passive image resolving sensor means in said image plane for providingpicture output signals indicative of the two-dimensional image of saidfield of view; laser means for generating a laser beam; scanning beamdeflecting means for deflecting said laser beam; LADAR sensor means;beam splitter means between said optical system means and said imageplane for directing said laser beam deflected by said scanning beamdeflecting means through said optical system means towards said objectto scan over the whole surface of said object and for directing lightreflected by said object and collected by said optical system meanstowards said LADAR sensor means for providing relief information foreach scanned spot of said object; image processing means to which saidpicture output signals from said passive image resolving sensor meansare applied for segmenting and detecting individual objects within saidfield of view, said image processing means being arranged to controlsaid scanning beam deflecting means such that said laser beam scanssubstantially only the surface of objects detected by said imageprocessing means within said field of view; wherein said scanning beamdeflecting means are controlled by said image processing means to scan,in a first scanning cycle, the objects detected by said passive imageresolving sensor along a path sweeping over the objects only once,whereby first scanning cycle information is obtained, said firstscanning cycle information is applied to selecting means for selecting,out of the objects detected by said passive image resolving sensor thoseobjects meeting predetermined criteria, and said scanning beamdeflecting means being controlled by said image processing means andsaid selecting means to scan, in a second scanning cycle, line-by-line,only limited areas of the field of view containing said selectedobjects; means for deriving, from said relief information, athree-dimensional relief representation of said object; and means foridentifying said object on the basis of said two-dimensional image andsaid three-dimensional relief representation.